<!DOCTYPE HTML>
<html lang="en" class="light" dir="ltr">
    <head>
        <!-- Book generated using mdBook -->
        <meta charset="UTF-8">
        <title>Tracing - The Rustup developer guide</title>


        <!-- Custom HTML head -->
        
        <meta name="description" content="">
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <meta name="theme-color" content="#ffffff">

        <link rel="icon" href="favicon.svg">
        <link rel="shortcut icon" href="favicon.png">
        <link rel="stylesheet" href="css/variables.css">
        <link rel="stylesheet" href="css/general.css">
        <link rel="stylesheet" href="css/chrome.css">
        <link rel="stylesheet" href="css/print.css" media="print">

        <!-- Fonts -->
        <link rel="stylesheet" href="FontAwesome/css/font-awesome.css">
        <link rel="stylesheet" href="fonts/fonts.css">

        <!-- Highlight.js Stylesheets -->
        <link rel="stylesheet" href="highlight.css">
        <link rel="stylesheet" href="tomorrow-night.css">
        <link rel="stylesheet" href="ayu-highlight.css">

        <!-- Custom theme stylesheets -->

    </head>
    <body class="sidebar-visible no-js">
    <div id="body-container">
        <!-- Provide site root to javascript -->
        <script>
            var path_to_root = "";
            var default_theme = window.matchMedia("(prefers-color-scheme: dark)").matches ? "navy" : "light";
        </script>

        <!-- Work around some values being stored in localStorage wrapped in quotes -->
        <script>
            try {
                var theme = localStorage.getItem('mdbook-theme');
                var sidebar = localStorage.getItem('mdbook-sidebar');

                if (theme.startsWith('"') && theme.endsWith('"')) {
                    localStorage.setItem('mdbook-theme', theme.slice(1, theme.length - 1));
                }

                if (sidebar.startsWith('"') && sidebar.endsWith('"')) {
                    localStorage.setItem('mdbook-sidebar', sidebar.slice(1, sidebar.length - 1));
                }
            } catch (e) { }
        </script>

        <!-- Set the theme before any content is loaded, prevents flash -->
        <script>
            var theme;
            try { theme = localStorage.getItem('mdbook-theme'); } catch(e) { }
            if (theme === null || theme === undefined) { theme = default_theme; }
            var html = document.querySelector('html');
            html.classList.remove('light')
            html.classList.add(theme);
            var body = document.querySelector('body');
            body.classList.remove('no-js')
            body.classList.add('js');
        </script>

        <input type="checkbox" id="sidebar-toggle-anchor" class="hidden">

        <!-- Hide / unhide sidebar before it is displayed -->
        <script>
            var body = document.querySelector('body');
            var sidebar = null;
            var sidebar_toggle = document.getElementById("sidebar-toggle-anchor");
            if (document.body.clientWidth >= 1080) {
                try { sidebar = localStorage.getItem('mdbook-sidebar'); } catch(e) { }
                sidebar = sidebar || 'visible';
            } else {
                sidebar = 'hidden';
            }
            sidebar_toggle.checked = sidebar === 'visible';
            body.classList.remove('sidebar-visible');
            body.classList.add("sidebar-" + sidebar);
        </script>

        <nav id="sidebar" class="sidebar" aria-label="Table of contents">
            <div class="sidebar-scrollbox">
                <ol class="chapter"><li class="chapter-item expanded "><a href="index.html"><strong aria-hidden="true">1.</strong> Introduction</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="linting.html"><strong aria-hidden="true">1.1.</strong> Linting</a></li></ol></li><li class="chapter-item expanded "><a href="coding-standards.html"><strong aria-hidden="true">2.</strong> Coding standards</a></li><li class="chapter-item expanded "><a href="version-numbers.html"><strong aria-hidden="true">3.</strong> Version numbers</a></li><li class="chapter-item expanded "><a href="release-process.html"><strong aria-hidden="true">4.</strong> Release process</a></li><li class="chapter-item expanded "><a href="tips-and-tricks.html"><strong aria-hidden="true">5.</strong> Tips and tricks</a></li><li class="chapter-item expanded "><a href="tracing.html" class="active"><strong aria-hidden="true">6.</strong> Tracing</a></li></ol>
            </div>
            <div id="sidebar-resize-handle" class="sidebar-resize-handle">
                <div class="sidebar-resize-indicator"></div>
            </div>
        </nav>

        <!-- Track and set sidebar scroll position -->
        <script>
            var sidebarScrollbox = document.querySelector('#sidebar .sidebar-scrollbox');
            sidebarScrollbox.addEventListener('click', function(e) {
                if (e.target.tagName === 'A') {
                    sessionStorage.setItem('sidebar-scroll', sidebarScrollbox.scrollTop);
                }
            }, { passive: true });
            var sidebarScrollTop = sessionStorage.getItem('sidebar-scroll');
            sessionStorage.removeItem('sidebar-scroll');
            if (sidebarScrollTop) {
                // preserve sidebar scroll position when navigating via links within sidebar
                sidebarScrollbox.scrollTop = sidebarScrollTop;
            } else {
                // scroll sidebar to current active section when navigating via "next/previous chapter" buttons
                var activeSection = document.querySelector('#sidebar .active');
                if (activeSection) {
                    activeSection.scrollIntoView({ block: 'center' });
                }
            }
        </script>

        <div id="page-wrapper" class="page-wrapper">

            <div class="page">
                                <div id="menu-bar-hover-placeholder"></div>
                <div id="menu-bar" class="menu-bar sticky">
                    <div class="left-buttons">
                        <label id="sidebar-toggle" class="icon-button" for="sidebar-toggle-anchor" title="Toggle Table of Contents" aria-label="Toggle Table of Contents" aria-controls="sidebar">
                            <i class="fa fa-bars"></i>
                        </label>
                        <button id="theme-toggle" class="icon-button" type="button" title="Change theme" aria-label="Change theme" aria-haspopup="true" aria-expanded="false" aria-controls="theme-list">
                            <i class="fa fa-paint-brush"></i>
                        </button>
                        <ul id="theme-list" class="theme-popup" aria-label="Themes" role="menu">
                            <li role="none"><button role="menuitem" class="theme" id="light">Light</button></li>
                            <li role="none"><button role="menuitem" class="theme" id="rust">Rust</button></li>
                            <li role="none"><button role="menuitem" class="theme" id="coal">Coal</button></li>
                            <li role="none"><button role="menuitem" class="theme" id="navy">Navy</button></li>
                            <li role="none"><button role="menuitem" class="theme" id="ayu">Ayu</button></li>
                        </ul>
                        <button id="search-toggle" class="icon-button" type="button" title="Search. (Shortkey: s)" aria-label="Toggle Searchbar" aria-expanded="false" aria-keyshortcuts="S" aria-controls="searchbar">
                            <i class="fa fa-search"></i>
                        </button>
                    </div>

                    <h1 class="menu-title">The Rustup developer guide</h1>

                    <div class="right-buttons">
                        <a href="print.html" title="Print this book" aria-label="Print this book">
                            <i id="print-button" class="fa fa-print"></i>
                        </a>
                        <a href="https://github.com/rust-lang/rustup/tree/master/doc/dev-guide" title="Git repository" aria-label="Git repository">
                            <i id="git-repository-button" class="fa fa-github"></i>
                        </a>
                        <a href="https://github.com/rust-lang/rustup/edit/master/doc/dev-guide/src/tracing.md" title="Suggest an edit" aria-label="Suggest an edit">
                            <i id="git-edit-button" class="fa fa-edit"></i>
                        </a>

                    </div>
                </div>

                <div id="search-wrapper" class="hidden">
                    <form id="searchbar-outer" class="searchbar-outer">
                        <input type="search" id="searchbar" name="searchbar" placeholder="Search this book ..." aria-controls="searchresults-outer" aria-describedby="searchresults-header">
                    </form>
                    <div id="searchresults-outer" class="searchresults-outer hidden">
                        <div id="searchresults-header" class="searchresults-header"></div>
                        <ul id="searchresults">
                        </ul>
                    </div>
                </div>

                <!-- Apply ARIA attributes after the sidebar and the sidebar toggle button are added to the DOM -->
                <script>
                    document.getElementById('sidebar-toggle').setAttribute('aria-expanded', sidebar === 'visible');
                    document.getElementById('sidebar').setAttribute('aria-hidden', sidebar !== 'visible');
                    Array.from(document.querySelectorAll('#sidebar a')).forEach(function(link) {
                        link.setAttribute('tabIndex', sidebar === 'visible' ? 0 : -1);
                    });
                </script>

                <div id="content" class="content">
                    <main>
                        <h1 id="tracing"><a class="header" href="#tracing">Tracing</a></h1>
<p>Similar to other tools in the Rust ecosystem like rustc and cargo,
rustup also provides observability/logging features via the <code>tracing</code> crate.</p>
<p>The verbosity of logs is controlled via the <code>RUSTUP_LOG</code> environment
variable using <code>tracing_subscriber</code>’s <a href="https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#directives">directive syntax</a>.</p>
<h2 id="console-based-tracing"><a class="header" href="#console-based-tracing">Console-based tracing</a></h2>
<p>A <code>tracing_subscriber</code> that prints log lines directly to <code>stderr</code> is directly
available in the prebuilt version of rustup since v1.28.0.</p>
<p>For historical reasons, if <code>RUSTUP_LOG</code> is not set, this subscriber will print
the log lines in a format that mimics the “legacy” <code>stderr</code> output in older
versions of rustup:</p>
<pre><code class="language-console">&gt; rustup default stable
info: using existing install for 'stable-aarch64-apple-darwin'
info: default toolchain set to 'stable-aarch64-apple-darwin'

  stable-aarch64-apple-darwin unchanged - rustc 1.79.0 (129f3b996 2024-06-10)
</code></pre>
<p>However, once <code>RUSTUP_LOG</code> is set to any value, rustup’s “custom logging mode” will
be activated, and <code>tracing_subscriber</code>’s builtin output format will be used instead:</p>
<pre><code class="language-console">&gt; RUSTUP_LOG=trace rustup default stable
2024-06-16T12:08:48.732894Z  INFO rustup::cli::common: using existing install for 'stable-aarch64-apple-darwin'
2024-06-16T12:08:48.739232Z  INFO rustup::cli::common: default toolchain set to 'stable-aarch64-apple-darwin'

  stable-aarch64-apple-darwin unchanged - rustc 1.79.0 (129f3b996 2024-06-10)
</code></pre>
<p>Please note that since <code>RUSTUP_LOG=trace</code> essentially accepts log lines from
all possible sources, you might sometimes see log lines coming from rustup’s
dependencies, such as <code>hyper_util</code> in the following example:</p>
<pre><code class="language-console">&gt; RUSTUP_LOG=trace rustup update
[..]
2024-06-16T12:12:45.569428Z TRACE hyper_util::client::legacy::client: http1 handshake complete, spawning background dispatcher task
2024-06-16T12:12:45.648682Z TRACE hyper_util::client::legacy::pool: pool dropped, dropping pooled (("https", static.rust-lang.org))

   stable-aarch64-apple-darwin unchanged - rustc 1.79.0 (129f3b996 2024-06-10)
  nightly-aarch64-apple-darwin unchanged - rustc 1.81.0-nightly (3cf924b93 2024-06-15)

2024-06-16T12:12:45.693350Z  INFO rustup::cli::rustup_mode: cleaning up downloads &amp; tmp directories
</code></pre>
<p>It is also possible to limit the sources of the log lines and the desired
max level for each source. For example, set <code>RUSTUP_LOG=rustup=DEBUG</code> to
receive log lines only from <code>rustup</code> itself with a max verbosity of <code>DEBUG</code>.</p>
<h2 id="opentelemetry-tracing"><a class="header" href="#opentelemetry-tracing">Opentelemetry tracing</a></h2>
<blockquote>
<p><strong>Prerequisites:</strong> Before following the instructions in this section,
<code>protoc</code> must be installed, which can be downloaded from GitHub
or installed via a package manager.</p>
</blockquote>
<p>The feature <code>otel</code> can be used when building rustup to turn on Opentelemetry
tracing with an OLTP GRPC exporter.</p>
<p>This can be very useful for diagnosing performance or correctness issues in more
complicated scenarios.</p>
<p>The normal <a href="https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md">OTLP environment
variables</a>
can be used to customise its behaviour, but often the simplest thing is to just
run a Jaeger docker container on the same host:</p>
<pre><code class="language-sh">docker run -d --name jaeger   -e COLLECTOR_ZIPKIN_HOST_PORT=:9411   -e COLLECTOR_OTLP_ENABLED=true   -p 6831:6831/udp   -p 6832:6832/udp   -p 5778:5778   -p 16686:16686   -p 4317:4317   -p 4318:4318   -p 14250:14250   -p 14268:14268   -p 14269:14269   -p 9411:9411   jaegertracing/all-in-one:latest
</code></pre>
<p>Then build <code>rustup-init</code> with tracing:</p>
<pre><code class="language-sh">cargo build --features=otel
</code></pre>
<p>Run the operation you want to analyze. For example, we can now run <code>rustup show</code> with tracing:</p>
<pre><code class="language-sh">RUSTUP_FORCE_ARG0="rustup" ./target/debug/rustup-init show
</code></pre>
<p>And <a href="http://localhost:16686/search?service=rustup">look in Jaeger for a trace</a>.</p>
<p>Tracing can also be used in tests to get a trace of the operations taken during the test.
To use this feature, build the project with <code>--features=otel,test</code>.</p>
<h3 id="adding-instrumentation"><a class="header" href="#adding-instrumentation">Adding instrumentation</a></h3>
<p>The <code>otel</code> feature uses conditional compilation to only add function instrument
when enabled. Instrumenting a currently uninstrumented function is mostly simply
done like so:</p>
<pre><pre class="playground"><code class="language-rust"><span class="boring">#![allow(unused)]
</span><span class="boring">fn main() {
</span>#[tracing::instrument(level = "trace", err(level = "trace"), skip_all)]
<span class="boring">}</span></code></pre></pre>
<p><code>skip_all</code> is not required, but some core structs don’t implement Debug yet, and
others have a lot of output in Debug : tracing adds some overheads, so keeping
spans lightweight can help avoid frequency bias in the results - where
parameters with large debug in frequently called functions show up as much
slower than they are.</p>
<p>Some good general heuristics:</p>
<ul>
<li>Do instrument slow blocking functions</li>
<li>Do instrument functions with many callers or that call many different things,
as these tend to help figure the puzzle of what-is-happening</li>
<li>Default to not instrumenting thin shim functions (or at least, only instrument
them temporarily while figuring out the shape of a problem)</li>
<li>Be way of debug build timing - release optimisations make a huge difference,
though debug is a lot faster to iterate on. If something isn’t a problem in
release don’t pay it too much heed in debug.</li>
</ul>

                    </main>

                    <nav class="nav-wrapper" aria-label="Page navigation">
                        <!-- Mobile navigation buttons -->
                            <a rel="prev" href="tips-and-tricks.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
                                <i class="fa fa-angle-left"></i>
                            </a>


                        <div style="clear: both"></div>
                    </nav>
                </div>
            </div>

            <nav class="nav-wide-wrapper" aria-label="Page navigation">
                    <a rel="prev" href="tips-and-tricks.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
                        <i class="fa fa-angle-left"></i>
                    </a>

            </nav>

        </div>




        <script>
            window.playground_copyable = true;
        </script>


        <script src="elasticlunr.min.js"></script>
        <script src="mark.min.js"></script>
        <script src="searcher.js"></script>

        <script src="clipboard.min.js"></script>
        <script src="highlight.js"></script>
        <script src="book.js"></script>

        <!-- Custom JS scripts -->


    </div>
    </body>
</html>
